Skip to content

Qwen3.5 FP4 GB300 disagg dynamo-sglang: bump image to v0.5.14-cu130 + NVFP4-V2 checkpoint#2238

Open
hshrivastava-droid wants to merge 2 commits into
mainfrom
qwen3.5-fp4-gb300-dynamo-sglang-v2-checkpoint
Open

Qwen3.5 FP4 GB300 disagg dynamo-sglang: bump image to v0.5.14-cu130 + NVFP4-V2 checkpoint#2238
hshrivastava-droid wants to merge 2 commits into
mainfrom
qwen3.5-fp4-gb300-dynamo-sglang-v2-checkpoint

Conversation

@hshrivastava-droid

Copy link
Copy Markdown
Collaborator

No description provided.

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

Comment thread perf-changelog.yaml
Comment on lines 4859 to +4868
description:
- "Add GB300 DeepSeek-V4 dynamo-sglang agentic recipes (agg-tp4 + dep8 disagg pareto)"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2157


- config-keys:
- qwen3.5-fp4-gb300-dynamo-sglang
description:
- "Bump SGLang image to v0.5.14-cu130 for the Qwen3.5 FP4 GB300 disaggregated dynamo-sglang config"
- "Switch to the nvidia/Qwen3.5-397B-A17B-NVFP4-V2 checkpoint (model pre-staged on cluster as Qwen3.5-397B-A17B-NVFP4-V2)"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2238

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 This PR strips the two trailing spaces from the pre-existing blank separator line in perf-changelog.yaml (the one PR #2157 left as \n) before appending the new entry, turning it into a bare empty line. AGENTS.md explicitly forbids this exact change ("Never delete or modify whitespace in perf-changelog.yaml ... Altering it breaks CI"), and it will fail the check-changelog CI gate. Fix by restoring the original trailing-space line byte-for-byte and appending the new entry after it.

Extended reasoning...

What's wrong: The base file (as of PR #2157, commit 3355132) ends with the byte sequence ...pull/2157\n \n — the blank separator line after the last entry carries two trailing spaces. This PR's diff shows that line changing from - to + (a bare empty line) before the new entry is appended:

   pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2157
-  
+
+- config-keys:
+    - qwen3.5-fp4-gb300-dynamo-sglang
...

Why it's forbidden: AGENTS.md is explicit and unambiguous on this point: "Never delete or modify whitespace in perf-changelog.yaml - CI depends on exact whitespace (including trailing spaces on blank separator lines). Altering it breaks CI." This diff does precisely the prohibited operation.

Why it will actually fail CI (not just a style nit): utils/validate_perf_changelog.py::validate_raw_change() is invoked by the check-changelog job in .github/workflows/run-sweep.yml (--base-ref origin/${{ github.base_ref }}). For any PR that only appends entries, it enforces:

if not head_raw.startswith(base_raw):
    raise ChangelogValidationError(
        "appended entries changed historical perf-changelog.yaml bytes; "
        "restore the base file byte-for-byte and append at the end"
    )

Step-by-step proof (verified directly against the repo history):

  1. git show 9af6bf5^:perf-changelog.yaml | tail -c 20b'renceX/pull/2157\n \n' — base ends with \n \n (two trailing spaces before EOF).
  2. git show 9af6bf5:perf-changelog.yaml (this PR's head) at the same offset reads ...pull/2157\n\n- config-keys:... — the trailing spaces are gone.
  3. A direct byte comparison (head_raw.startswith(base_raw)) confirms this is False: the first divergence is at byte offset 255901, where base has pull/2157\n \n and head has pull/2157\n\n- config-.
  4. Because additions is nonzero for this PR (a new - config-keys: block was appended), validate_raw_change() takes the if not head_raw.startswith(base_raw) branch and raises ChangelogValidationError, which fails the check-changelog job — and downstream jobs (needs: check-changelog) gate on its success.

The fix: Keep the original last line of the base file exactly as-is ( — two trailing spaces, no more, no less) and append the new entry as a new block after it, so head_raw starts with base_raw byte-for-byte. Concretely, restore the line to before the new - config-keys: block rather than collapsing it to an empty line.

image: lmsysorg/sglang:nightly-dev-cu13-20260624-b2c8f7a2
model: nvidia/Qwen3.5-397B-A17B-NVFP4
image: lmsysorg/sglang:v0.5.14-cu130
model: nvidia/Qwen3.5-397B-A17B-NVFP4-V2

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is difference between nvidia/Qwen3.5-397B-A17B-NVFP4 & nvidia/Qwen3.5-397B-A17B-NVFP4-v2

@functionstackx functionstackx left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are yall seeing lots of customer traction for qwen3.5 hence why modelopt team has pritotized and used their limited engineering bandwidth on an v2 quant on qwen3.5?

+viz @Ankur-singh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants